Local Communication Protocols for Learning Complex Swarm Behaviors with Deep Reinforcement Learning

Swarm systems constitute a challenging problem for reinforcement learning (RL) as the algorithm needs to learn decentralized control policies that can cope with limited local sensing and communication abilities of the agents. While it is often difficult to directly define the behavior of the agents, simple communication protocols can be defined more easily using prior knowledge about the given task. In this paper, we propose a number of simple communication protocols that can be exploited by deep reinforcement learning to find decentralized control policies in a multi-robot swarm environment. The protocols are based on histograms that encode the local neighborhood relations of the agents and can also transmit task-specific information, such as the shortest distance and direction to a desired target. In our framework, we use an adaptation of Trust Region Policy Optimization to learn complex collaborative tasks, such as formation building and building a communication link. We evaluate our findings in a simulated 2D-physics environment, and compare the implications of different communication protocols.Comment: 13 pages, 4 figures, version 2, accepted at ANTS 201

The blockchain: a new framework for robotic swarm systems

Swarms of robots will revolutionize many industrial applications, from targeted material delivery to precision farming. However, several of the heterogeneous characteristics that make them ideal for certain future applications --- robot autonomy, decentralized control, collective emergent behavior, etc. --- hinder the evolution of the technology from academic institutions to real-world problems. Blockchain, an emerging technology originated in the Bitcoin field, demonstrates that by combining peer-to-peer networks with cryptographic algorithms a group of agents can reach an agreement on a particular state of affairs and record that agreement without the need for a controlling authority. The combination of blockchain with other distributed systems, such as robotic swarm systems, can provide the necessary capabilities to make robotic swarm operations more secure, autonomous, flexible and even profitable. This work explains how blockchain technology can provide innovative solutions to four emergent issues in the swarm robotics research field. New security, decision making, behavior differentiation and business models for swarm robotic systems are described by providing case scenarios and examples. Finally, limitations and possible future problems that arise from the combination of these two technologies are described

Cooperative object transport with a swarm of e-puck robots: robustness and scalability of evolved collective strategies

Cooperative object transport in distributed multi-robot systems requires the coordination and synchronisation of pushing/pulling forces by a group of autonomous robots in order to transport items that cannot be transported by a single agent. The results of this study show that fairly robust and scalable collective transport strategies can be generated by robots equipped with a relatively simple sensory apparatus (i.e. no force sensors and no devices for direct communication). In the experiments described in this paper, homogeneous groups of physical e-puck robots are required to coordinate and synchronise their actions in order to transport a heavy rectangular cuboid object as far as possible from its starting position to an arbitrary direction. The robots are controlled by dynamic neural networks synthesised using evolutionary computation techniques. The best evolved controller demonstrates an effective group transport strategy that is robust to variability in the physical characteristics of the object (i.e. object mass and size of the longest object’s side) and scalable to different group sizes. To run these experiments, we designed, built, and mounted on the robots a new sensor that returns the agents’ displacement on a 2D plane. The study shows that the feedback generated by the robots’ sensors relative to the object’s movement is sufficient to allow the robots to coordinate their efforts and to sustain the transports for an extended period of time. By extensively analysing successful behavioural strategies, we illustrate the nature of the operational mechanisms underpinning the coordination and synchronisation of actions during group transport

HE-LHC: The High-Energy Large Hadron Collider – Future Circular Collider Conceptual Design Report Volume 4

In response to the 2013 Update of the European Strategy for Particle Physics (EPPSU), the Future Circular Collider (FCC) study was launched as a world-wide international collaboration hosted by CERN. The FCC study covered an energy-frontier hadron collider (FCC-hh), a highest-luminosity high-energy lepton collider (FCC-ee), the corresponding 100 km tunnel infrastructure, as well as the physics opportunities of these two colliders, and a high-energy LHC, based on FCC-hh technology. This document constitutes the third volume of the FCC Conceptual Design Report, devoted to the hadron collider FCC-hh. It summarizes the FCC-hh physics discovery opportunities, presents the FCC-hh accelerator design, performance reach, and staged operation plan, discusses the underlying technologies, the civil engineering and technical infrastructure, and also sketches a possible implementation. Combining ingredients from the Large Hadron Collider (LHC), the high-luminosity LHC upgrade and adding novel technologies and approaches, the FCC-hh design aims at significantly extending the energy frontier to 100 TeV. Its unprecedented centre-of-mass collision energy will make the FCC-hh a unique instrument to explore physics beyond the Standard Model, offering great direct sensitivity to new physics and discoveries

Rogue waves of the Kundu-Eckhaus equation in a chaotic wave field

In this paper we study the properties of the chaotic wave fields generated in the frame of the Kundu-Eckhaus equation (KEE). Modulation instability results in a chaotic wave field which exhibits small-scale filaments with a free propagation constant, k. The average velocity of the filaments is approximately given by the average group velocity calculated from the dispersion relation for the plane-wave solution however direction of propagation is controlled by the $\beta$ parameter, the constant in front of the Raman-effect term. We have also calculated the probabilities of the rogue wave occurrence for various values of propagation constant k and showed that the probability of rogue wave occurrence depends on k. Additionally, we have showed that the probability of rogue wave occurrence significantly depends on the quintic and the Raman-effect nonlinear terms of the KEE. Statistical comparisons between the KEE and the cubic nonlinear Schrodinger equation have also been presented

Semi-dynamic modeling of tactical level land combat

We present a semi-dynamic approach to land combat modeling at tactical level. Our approach decomposes a battle into stages and mini battles. For each mini battle in each stage, we use three models: a mathematical programming model for optimizing force allocations, a Lanchester simulation model for predicting whether or not the stage targets are reached under the allocations, and a weapon effectiveness update model from one stage to the next. These models interact with each other within the framework of a decision support system to help the user with allocation decisions as well as prediction of force size and weapon requirements to win the battle

Cooperative Pollution Source Exploration and Cleanup with a Bio-inspired Swarm Robot Aggregation

© 2021, ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering.Using robots for exploration of extreme and hazardous environments has the potential to significantly improve human safety. For example, robotic solutions can be deployed to find the source of a chemical leakage and clean the contaminated area. This paper demonstrates a proof-of-concept bio-inspired exploration method using a swarm robotic system based on a combination of two bio-inspired behaviors: aggregation, and pheromone tracking. The main idea of the work presented is to follow pheromone trails to find the source of a chemical leakage and then carry out a decontamination task by aggregating at the critical zone. Using experiments conducted by a simulated model of a Mona robot, the effects of population size and robot speed on the ability of the swarm was evaluated in a decontamination task. The results indicate the feasibility of deploying robotic swarms in an exploration and cleaning task in an extreme environment